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ABSTRACT ' 

A program for trained vocational education curriculum 
specialists (VECS), consisting of 16 modules, was written, revised, 
and field tested at 15 sites nationwide. The instructional materials 
were written to deliver the highest rated competencies based on a 
f ield/-survey of vocational educators and review by a national 
advisory panel of vocational education experts. VECS modules were 
designed to create or upgrade an individual's vocational education 
curriculum development and management skills* Additional materials 
developed, were a guide for instructors and administrators and audio 
cassette tape for orienting potential users. For- the field test a t 
modified quasi-experimental/ pretest/posttest , treatment 
group/control group design was used. Participants were undergraduates 
in vocational education teacher preparation, practicing vocational 
educators, and persons with occupational skills who wished to teach 
their specialty at 12 Colleges/universities and two state departments 
of education. Field test evaluation forms were developed to measure- 
cognitive and affective outcomes and to collect biographical 
infoxraation. Results of the national field test demonstrated that the 
modules increased knowledge of topics necessary to the successful 
performance of skills central to the VECS role. They also t^nded to 
increase peoples' confidence in their ability to perform these 
skills. (A list of materials produced is appended; a summary report 
is available as C£ -031 802.) (YLB) 
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specialist training programs. . In 1974, the Bureau of .Occupational and f 
Adult Education (BOAE) awarded conttacp to the' American Institutes for 
Research (AIR) and to Washington State University (WSU) to produce, test,^ r- 
Sind revise such materials., In 1978, BOAE again contracted with AIR to con- 
duct a national field test of the VECS materials f* The overall goal of this 
project was to integrate the original two sets of materials, systematically 
field t,est them, and .encourage their -use in the field. 

In designing the first sets of materials, careful analyses of the com- 
petencj.es r&quired of a VECS were conducted based on a field survey of 
vocational educators and a review by a national advisory panel of voca- 
tional education experts. The instructional materials were written to 
.deliver* the highest rated competencies. The resulting materials weri pilot 
tested .at five universities by an independent third-party evaluator, and 
subsequently revised. AIR assumed that the competency base for the fitst 
sets of V£CS materisl$was sound. Thus, efforts in the national field test 
cone entrateT7m-£Trkegr£t:j.ng. and updating the first AIR and WSU materials 

J 

and on assuring that the resulting materia^ were appropriate for use in a 
wide Variety of settings with varied target populations. Efforts in this 
third revision cycle-^rior to the national field test improved strategies 
for presenti^inf ormatio^, added learning activities and support materi- 
als^ eljminsuted overlap and redundancy, and focused on current national 
* Vrtoritles rVgaVrffrig 4 seVfcqultjTari^ tttlttlflg *f Ar 1«Hi±cap$ed-afid- older* — 
students. ' Based on suggestions, from instructors, students, and consultants 
collected during the national field test, the modules were again revised 
and improved. 
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Purpose of the Modules r s y 

The VECS modules were designecLto create or upgrade an individual's 
vocational education curriculum development and management skills. These 

skills enable the curriculum specialist , to : (1) describe current issues in 

j 
/ 

vocational education* funding and governance; (2) design or" modify voca-^ 
tional education programs to provide for individual differences, rae'et the 
needs of special students, or meet labor market demands; (3) select curri- 
cular approaches, goals and objectives, and instructional strategies for 
vocational education programs; (4) prepare instructional materials; 
(5) conduct evaluations of vocational curricula; (6) manage a vocational 
cla^srooip or program; (7 > facilitate curriculum change; and (8) promote 
professional growth and staff development. 

Description of the Modules 
Since the VECS modules are intended to serve a broad purpose, their 
design had to be highly flexible. They synthesize an extensive amount of 

r . c 

information into a concise format that is organized to, promote efficient 



learning under a variety of - circumstances . The titles of the VECS modules 



are shown in" Table I. They can be. divided into three series of approxi- 

l 

mately equal length. The goal of the introductory series is to provide am 
historical ^ackground of the development of vocational ed^ication, present 7 
an overview of its scope, organization, and priorities, and describe the 
role and functions of the vocational education curriculum specialist. The * 

second series covers topics directly related to the preparation of instruc- 

— • • * * 

tional materials. Its goals are to teach vocational needs assessment and 
task analysis, specification of objectives , selection of instructional stra- , 
tegies, assessment of student achievement, ^nd^the seleptio^n or development 

' , -3- ' ' ^ ' 



K • • * 

of curriculum materials. The final series is designed to provide the cur- 
riculum specialist with administrative skills. Theae modules are concerned 
with evaluation, program management, curriculum inn6Vation, and staff * 
development . 1 \ 



TABLE I 
'VECS Module Titles 



L 

L 

Iacrocuctorv 3 
Series * , 

5 
6 



Vocational Educators and Curriculum Management 
The Scope of Vocational Education 
Organization of Vocational Education 
Legislative Mandates for Vocational Education 
Priorities in Vocational Education 

Vocational Education for Students vith Soej^Lal Needs 



Vocational Needs Assessment and Curriculum Deyeiooment 
Conducting Task Analyses and Developing Instructional Objectives 
Selecting Instructional Strategies and Assessing Student Achievement 
10. Relating Learning Differences and Instructional Methods 
11. Selecting and Preparing Instructional Materials 

12. Evaluating Vocational Education Curricula * «. 

13. Conducting Follow-Up Studies and Communicating Evaluation Results 
Administrative * ^ 
Series 14. Managing Vocational Education Programs * 

15. Preparing for Curriculum Change ^ * 

16. Staff Development ' 



Curriculum 

Development 

Series 
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Modules average about 7 5 pages in length and require about 30 to 50 

hows to'complete. 'They follow a standard ^format including a detailed list 

0 ' \ 

of behavioral gpals and objectives, and text, learning activities, and 

self -assessment items related to each goal. The modules usually contain 
from three gfo five goals each. Lear activities include reading in 
published resources, group projects, ^Rf discussion questions. Interviews 
with vocational educators and actual practice in curriculum development 
skills are frequently called for. The whole set of modules can be insti- 
tuted as the foundation of a formal curriculum specialis.t_Lrainlng._prQgram-' — 



8 



• * 

or selected gro'ups of modules can surve as, the focus of specific courses, . 
In addition, components of modules, individual modules, or series of mod- 
ules can be integrated into existing courses or programs. Because the^mod- 
ules are organized around specific goals and include alternative learning \ 
activities, their use is appropriate in traditional and nontraditional 
classrooms, inservice education programs, and independent study, 

& * ' 

Additional Materials s 
Using the^VECS modules: A guide for instructors and .administrators 
was written te> help vocational educators set up and manage professional 
development programs that focus on curriculum skills. The Guide presents 
instructional strategies appropriate for the many different situations in 
which the modules can be used. General guidelines, specif ic suggestions, 
and ideas to stimulate creativity are tiased on the experiences of field 
test instructors. The Guide identifies courses^-and programs^Tnt;o which, the 
modules might be incorporated, describes how faculty mighty be encouraged to 
try them, and discusses stegs in planning and initiating a new curriculum 
specialist training program. A list of the resources for each module that 
should be ordered t from their respective publishers is included. 



An audio cassette tape was also developed for /Orienting potential 
us^s v to the VECS modules. The tapeypresents interviews with field test 
instructors and sit£ coordinators who offer their advice on module imple- 



mentation. 
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Chapter II 

' FIELD TEST OF THE MATERIALS % 

Study of the VECS modules produces significant gains iri knowledge of 

the history and philosophy of vocational education and topi-cs in vocational 

education curriculum development , management } and evaluation. Use of the 

/* • 
modules also increases students 1 self-perception of personal competence in 

\ 

actiyities performed by 3 VECS. 



Field Test Design x 
A modified quasi-experimenta^ty. pretest/posttest , treatment group/con- 




trol group design was used. The design featured multiple replications with 
different types of students using various patterns of module implementation 
and methods of teaching. The field test was conducted under the types- of 

A 

conditions expected to represent actual module use, which precluded the 

type of control needed to carry out a scientifically rigorous research 

* * 
deSign. ' * 

< 

Sites were selected based on the appropriateness of the setting and 
the student population, and on their willingness to cooperate in the field 
test/ A local coordinator at each site identified instructors arid students 
to serve in the treatment and control* groups . An AIR staff member con- 
ducted orientation sessions at each sitef For the most part, intact 

classes were used, and instructors taught the modules following the general 

. **> 

guidelines established during the orientation session. Student partiqj.- 

/\ ' ' * . v • f 

pants were not paid. To maximize the likelihood of obt^ining^|^b^e data, 

fieJLd test data collection instruments were kept short and administer e d to 



~ 7 ~ 10 



\ 



control students only one -time. Because of the "naturalistic" character of 
the field test, site coordinators and AIR staff could noT*e#ercise strict • 
control over the schedule of module use, the method of module implementa- 
tj.on, or the selection and testing of treatment and control groups. 

TABLE 
- Module Groups 



* 5 






N'uober of 








Modules* 


Tooics ' 


Group 


L 


5 


f • * 

hiscory and philosophy or vocacional educacion % 


Group 


2 


5 


vocational .education curriculum development 


Group 


3. 


2 


procedures tor individualizing curricula for special students 


Grouo 


5 


• 5 


evaluation and administration of vocational education 



*Afce# che field cest, modules in Group 3 were divided becveen Groups L and - resulting in # 
che chree series of modules described earlier, One> module in Group 2 ^vas deleted and its 
significant content was Integraced into several of che ocher modules, resulting in che rinai 
cocal of 16 modules. 



For purposes of the analysis, the modules were divided' into four 

groups as sho>m in Table II.' Module groups 1 and 2 were taught in the fall 

of 197"9. All module groups were taught during the winter and spring of 

1980, although groups 3 and 4 were emphasized. Typically, students studied 

^ three to five modules each. Treatmeti£ students were tested both before and 

after they studied the particular modules taught, in their classes. Control 

students were tested onXy once. At some sit£s, controls were tested when 

treatment students were pretested;' at 6ther sites, when treatment students 

were ^osttested. The test results of only those students who 'studied all 

modules in a group and took both a pretest and a posttest were used in the 
v ■ - . # . 

Or * < 1 

analysis. Separate t-Jtests were calculated for each module group. 



/ 
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Field Test Sites and Participants 



*The sponsors of the 15 VECS field test sites -included twelve colleges 
or universities and two state departments of education. Five of the educa- 
tional institutions held pi^tsses on campus, six tased off-campus locations 
such as area vocational schools, and one university sponsored two sj^tes, 
pne ,bn campus and one at a military base. VECS instructors were most often 
^faculty, but also included a director of a regio^l occupational program, a 
principal of an area vocational school* and a director of a state curricu- 
lum and instructi6nal materials ceiiter. 

• j. . 

* Students who participated^tin the field tesf included: (1) undergradu- 

r 

ates preparing to be vocational education teachers; (2) practicing voca- • 
tional education teachers, administrators, counselors, and curriculum 
developers; and (3) persons with occupatipnal skills developed in business 
or the military who wished to teach their specialty.* Students' goals 
included obtaining a state vocational teaching certificate, a college 
degree at the undergraduate, or graduate level > and inservice professional 
development. Weekly classes, held during the day, at night, or on week- 
ends, were the T-aain* instructional delivery method, although several stu- 
fc detits took the modules through independent study ^arrangements. Class sizes 
'ranged from over 30 to under 10. 



Table III summarizes information on the field^test sites and partici- 
pants who supplied the data reported here. Data from all sites and all 

y It 

\ / participants" are' not reported due to the decision to base the analysis on 
results from only those students who studied all modules in a group. - 
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• 
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l.ocat ion: 
on or 


Purpose of 
Institution* 
Preservlce (P) 


Student 


„ Number of Students Who 

Contributed Data to the , 
Analysis^k by Module Group 








Spunaor 


off campus 


orlnservice (1) 


Character iat ics 


Croup 1 


Cropp 2 


Croup 3 


Croup 4 




* 


l. 

2. 


University 'of fdaho 
Washington State University 


off 


I 
P 


Vocational teachers 

Vul lire unrMt'lnnal teachers 


5 




7 


7 


V 




3. 


Ca 1 i forn ia« State University, 
Sacramento 


'off 


P 


Occupational specialists 




8 


if* 








4. Virginia Polytechnic Institute 
/and State University 


Oil 


I 


^ApniuLir'u am\ iwm f~ Ht » i*nn<l.'i r v Lejicliery 

I 


10 


8 




- _ 






5. 


Southern Illinois University, 
Carbondale . 4 


on 


P 


Occupational specialists and future 
business education teachers 


— 


25 


— 


9 








Oklahoma State Department of Voca- 
tlunal and Technical Education 


N/A 


I 


Staff of curriculum and instructional 
materials center 
. * / 


7 


* . 6 


18 


18 






7. 


Oklahoma State University 


on 


I 


Voca tional teachers 




5 


10 


10 




* 1 

o * 


8. 


Southern Illinois University, 
Carbondale at Travis Air Force 
Base 


off 




Military occupational specialists 


W 23 












9. 


University of South Florida 


m 

off 


I 


v 

Vocational teachers, supervisors, 
and a dm i u is t rat or s 


— 




— 


• 


\ 




10. 


New Jersey Department of- * 
Education 


N/A 


I 


Vocational teachers, coordinators, 

nn/l (111 1 (bmr>A cnnntti'lnrQ ' 


— 


— 






1 


11. 


J^UNY College of 'Technology, 
Utica/Knrae. 


on 


P & I 


Craduate^students with vocational- 
\. cciifi ilu 1 pucK^r utinuo 


2 





« 










University of North Carolina, 
Greensboro 


on 


I 


Home economics teachers 

t * ■ 


5 


13 


4 T- 






• 


n. 


North Carolina State University 


on 


I 


Touchers and state department admin- 
istrators 


11 


2 










14. 


Texas A & M University 


off 


I 


Vocational teachers, adminisr rators , 
and counselors * ■* 


16 


10 


7 


8 




13 


}5. 


Kast Texas State University 


off 


I 


» 9* 

Vocational teachers and adisi in 1 tit ra- 
tors 1 ' 


26 


12 


17 


32 


14 














109 


93 




84 
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Measurement of Effect 
* % ' — 

Socially constructed * instruments for use in the field test were r 
developed to estimate the ef feitiveness of the VECS modules. Field test 
evaluation forms (FTEFs*) measured the cognitive and affecti-ve outcomes of 
module study and collected biographical information on treatment and con- 
trol students. The FTEFs provided information on the overall" effect of 
studying groups of modu les. They were not designed to test mastery of each 
objective of each module goal. Effects of module groups 1 and 2 were 
assessed by one set of FTEFs, while module groups 3 ancT4 were tested by 
another set. ^ " 1 

The FTEFs had six s-ections, each designed to collect a different kind , 

.of information. One section requested background information on education, 

current occupation, and previous worK experience. Another section provided 

v 

a checklist on which respondents indicated ttecent professional development 
experiences. Two sections requested participants to rate themselves on . 
twelve activities that are typical of the VECS role. The first self-rating 
was of a persoa's competence in performing the activities; the second was 
of a person's desire to engage in the activities. Competence self-ratings 

ranged from 1-6; attitude self-ratings ranged from 1-4. 

» 

1 

The remaining two sections contained multiple-choice and shortranswer 
items based on significant cognitive outcomes of the modules. In order to 
reduce response burden and increase the number of people who might respond, 
the number of items included on any one FTEF was severely restricted. Only 
one, four-alternative, multiple-choice question was included for each goal 
of each module included in the gro.up of modules tested by a form. The set 

-il- 
ls 



of FTEFs testing' module groups 1 and 2 contained 37 multiple-choice items; 

' S * ' L 

•FTEFs for module groups 3 and 4 contained 23 multiple-choice jL terns. Two 
sets of parallel multiple-choice Items were written for each module group. 
Treatment students* took one §et of items as a pretest and the other set as 
a posttest. In-order to control for possible differences in the difficulty 



pf the two parallel sets, on& set was used as a pretest at about half the 
sites, while* the other set was included in the pretest at the remaining . 
sites. About half the control pretests and half the control posttests con- 
tained each set of multiple-choice items, as well. 

. • • • * ■ T : i r- 

One short-answer 'item wa^ written for each of the modules in groups 
2-4. As these items were intended toassess cognitive prerequisites of the 

perf o^mancfe o'f skills central to the VECS role, fio short-ajiswer items were 

- • * 

included to test '.outcomes of the group 1 modules because these modules pro- 
vide background information tfnd do not'focus on skills. The short-answer 
skill items required students to list such things as the appropriate proce- 
dures for certain situations ,^ the sequential st<eps in a procedure, „or the 
advantages and disadvantaged of several alternative procedures.. Responses* 
were rated on a scale of 1-5 based on pre-specif ied criteria that reflected 
the quality of , the response in teifms of its accuracy anc^completeness • 

JSl^ort-answer items were not included in the FTEFs administer^! to 
control group* students. Because the short-answer items were so directly 
related to the content pf the modules, it was felt that asking control 

; 4 

' it* 

students to answer them without the benefit of module study would be 
excessively burdensome ^n relation to the value of the data that would be^ 
obtained'. . 

*, 

■\ * 
-12- 
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InterpretabjLlity of Measures « - 
The FTEFs were prepared according to a careflil, step-by-step develop- 
ment process, and were- approved by the Federal Education Data Acquisition 
Council (FEDAC), the group -charged with ensuring that data are collected by 
the most efficient an£ effective taean&Z The statements .of VECS 1 activities 
used in the self-assessments of '^competence and attitude were derived from 
1 the content of the pjdulei, which was in turn derived # from the detailed 
competency analysis ^performed prior to the writing' of the initial set of 
modules. The multiple-choice^nd performance items were written based on 
objectives and topicA the modules' authors considered most significant. 

/ ^ 

Each it^m was reviewed by tfye module's author for content accuracy and by 
the project's evaluation director for technical adequacy. Items were then 
revised as many times as^necessary . 

The multiple-choice 'items were pilot-tested. Respondents answered, 

* « 

critiqued, and edited' the items. Point-biserial correlations and diffi- 
culty levels were calculated for each item. Items with low correlations or 
very high qr very^low difficulty levels were removed or improved. An 
attempt was made to balance difficulty levels of multiple-choice item sets 

w * 

when, constructing the alternative foftos of the FTEFs. However, we could 
not o-btain clear evidence that parallel' multiple-choice i'tem sets were of 
[equal difficulty. Therefore, cases were dropped randomly from the analysis 
so that equal numbers or proportions of people in the groups compared took 
each set of items. 



s 



ERIC 



Spearman-Brown estimates of the split-hajf reliabilities of the two 
parallel multiple-choice item sets were calculated for each of the four 



module groups. 0 The resulting eight correlations ranged from .12 to .55 

with approximately .40 the mode and .33 the mean. «,Two decisi6ns made in' 

designing the FTEFs operated against obtaining high reliability estimates. 
* 

First the number of multiple-choice i.teras included on any one/ form was' 
deliberately kept low. The numbers of items for module groups 1-A (see 
Table II) were 17, 20, 7 and 16 respectively. The SpeaYman-Brown reli- 
ability estimate for a 60-item test with the same split-half correlation as 
the 17 items written for module group 1 would be effectively double the 
actual estimate obtained. The second decision operatir\g against obtaining 
high reliability estimates concerned using one set of items to test the 
content of % a number of modules. The, items testing a group of modules did 
not include interchangeable measures of the same learning, and since the 
S peatman -Brown statistic essentially is a measure of internal consistency, 
*a high reliability estimate could not be expected, 

/ ' ■ , 

Despite the low reliabilities calculated for the multiple-choice^ 

items, two points should^bje kept in mind. Firsi:, the content of the* test 

items was judged valid by the motlule authors and the project director, 

thase persons most familiar with the concepts 'the ^odules-were designed to 

each.*. Second, the FTEF& were the only instruments available to assess the 

* > 

effects of the modules. No standardized tests existed that measured the 

"V * w * , * > f 

gdtfls of these modules in a systematic manner, 
i • 

\ ■ 

Credibility of Evidence 
An attempt was made to ensure that scoring and analysis was done 
objectively and reliably. While the tests tha,t provided -the data upon 
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which this submission is based were administered by the teachers of treat- 
ment ^and control students in their classrooms, the completed tests were 

sent directly* to AIR where objective methods were employed for scoring. 

/ 

Data were coded and keytaped by clerical staff who knew little about the 
nature of the field test. Computer services staff of AIR, rather than 
project staff, analyzed the data by using standard statistical packages. 



\ 
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. . Chapter III 

f ' RESULTS OF THE FIELD TEST . 

4 

The main- effects claimed for the VECS modules are cognitive and the ^ 
» , 

claims rest on the results obtained on the multiple-choice and short-answer 

A * 

itemfc. 'The cognitive effects are reinforced by an affective oCitcome 
revealed in'the self-ratings of competence. The assertion of the effec- 
tiveness of the VECS modules is based primarily*. on the comparison of the 
pretest and posttest results of students who studied the modules. Although 
some treatment ^roup/cqntrol group analyses were performed and are* reported 
they are included in this submission as secondary support for the assertion 

Treatment £roup pretest/posttest comparisons are 'appropriate for show- 
ing that newly developed instructional materials do, in fact* produce the 
kind of results for which they were designed. For the VECS field test, 
control* group data were not intended to demonstrate that the modules are* 
more effective thaa a competing treatment (there ^is none), but rather to 
indicate that factors other than module study occurring during the course 
of the field, test were unlikely to have produced the obtained results • 
Factors iq the design of the field test that are discussed later, as well 

a^ the control group data, indicate it is likely that the field test 

{. 

results were indeed produced by iao3aj Le" st ody . 



s 

Data Analysis and Results 
f * In preparation for the analysis, several > summary scores were created. 
Module group scores on multiple-choice items were calculated for control 
students, and for treatment students who studied all modules in a group. 

'/ 



Similar scqres for treatment students were created for short-answer items. 
(As discussed earlier, control students' FTEFs did not contain short-answer 
items.) Summary scores were created for the self-ratings *of competence and 
attitude by additig together the ratings made on- each of the 12 activity ^ 
statements. The total number of professional development experiences 
reported was also summarized in a^score. 
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In general, two types of analyses fdr each module group jwere conducted. 
First, treatment group students 1 'pre test results were compared with their 
posttest results using t-tests for paired samples. Second, t-tests for 
independent samples were used to compare the results obtained from post- 
testeii trea^Q^nt, students to the results obtained from control students who. i 
were tested at the same time* Treatment pre- v.ersus treatment post- com- 
parisons were analyzed for multiple-choice and short-answer item module 

* * * ' • <. 

grdup scores, and competence and attitude self-rating summary scores. 

Treatment group/control group comparisons were conducted using l multiple- 

ch&ice item module group scores, competence and attitude Self-rating * 

summary scores, and professional development experience mtmnrary scores. 

Data from the comparison^of the treatment group's pretest and post- 
test results are shown-in Table IV. Multiple-choice item comparisons for 

o * 

— V 

all but one of the module groups show gains significant at the .05 level or 
better. The remaining comparison approached significance (p».058). Since 
this group is composed of only two modules, the small number ^of items (7) 
probably, accounted for the lack of a significant gain. Comparisons of 
results on ^hort-answer items for all module groups were significant at 
better* than the .0001 level. Comparisons of competence self-ratings for 
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TABLE Iv" 

Results of Treatment Group Pretest vs. Treatment Group Posttest Paired Comparisons ' 



Number of 
Subjects 



Highest 
Possible, 
Score 




Mean 



Standard' 
Deviation 



Module Group 1 
t - 4.28 d 


Pretest 
Posttest 


96 
96 


17 
17 




7.08 " 
8.29 




2.30 
2.44 


Module Group 2 
: - 3.75c 


Pretest 
Posttest ' 


78 
78 


20 
20 




7.74 
9.09* 




2.67 
2.65 


Module Group 3 
c - 1.92 


Pretest 
Posttest 


69 
69 


7 
7 




3.10 
3.65 




1.30 
1.84 . 


Module Group 4 
c - 2.23 a 


Pretest 
Posttest 

^ 


68 
63 


16 
16 




7.78 
8.63 




2.33 
3.19 


















Module Group 2 
: - 6.90 d 


Pretest 
Posters 


77 

77 


25 
25 




5.12 
8.64 




" 3.50 
4.68 


Mocjule Group 3 
c 3*1. 97* , 


Pretest 
?osttes"t 


69 
69 


10 
10 




2.97 
5.33 




2.15 
3.37 


Module Group 4 
z - 6.19 d 


Pretest 1 
Posttest 


68 
68 


25 
25 




3.88* 
8.74 




2.99 
?,68 



?e8u.lZ3 on 5g If^F&tings of Cony ete nee 



Module Group 1 
t » 6.14 d _ 

Module Group 2 
* * c - 3.31 d 


Pretest 
Posttest 

Pretest / 
- Posttest/ 


89 

89 

69 
69 




4 72 
72 

72 
72 




1 


42.13 
47.93 

39.62 
49.46 


11.72 
9.33 

11.06 
8.34 


» Module Group 3 
t * 2.20 a 


Pretest 
Posttest 


65 
, 65 




72 
72 






43.35 
46.52 


10.84 
10.16 


*3odule Group 4 
c - 2.90 b 


Pretest 
Posttest 


60 

60 * 




72 
72 






42.07 
45.60 


11.44 
11.56 


i. Results- on 5 


elf-Razinca of Attitude 
















Module Groups 1 

t - -o;4i 


1 Pretest 
Posttest 


80 • 
00 




48 
48 






36.93 
36.69 


4.26 
4.34 


Module Group 2 
c • -0.81 


4 Pretest 
Posttest 


57 
57. 




48 
48 




% 


38.14 
37.39 


5,94 


^todule Gr,oup 3 
t - 2.0l a 


Pretest 
Posttest 


49 
49 




48 
48 






34,53 
36.06 


6.03 
4.64 


Module Group 4 
t - 0.62 


Pretest 

Posttest # 


54 
54 




48 
48 


t 




33.28 V 
33.69 • 


6.23 
7.46 


^Significant at 
Significant at 


the .05 level 
the .01 level 




^Significant at 
Significant at 


the 
the 


.00L 
.0001 


level 
Ifctrel 
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all module groups showed significant gains, but those for attitude were 
inconclusive • > * ; 

. - 1 •> ~~ 

< For the treatment group/control group comparisons, it was expected 
that the data from pretested and posttested controlsjcould.be combined. 
However, preliminary analyses showed that the two control groups scored 
significantly differently on multiple-choice items * Therefore, data from 
the two control groups were not combined. Only the results from controls 
who were testfed at the 'time treatment students^ were .posttested were com- 
pared'with treatment students' posttest results. Unfortunately, treatment 
% group/control group comparisons could be calculated only £d!'r module groups x 
1 and 2. Based on their scores on the multiple-choice items, it appears 
that the control students* for module groups 3 and* 4 were a l^ss able group 
than the treatment students, thus a treatment group/control group compari- 
son u^ing their scores would have ovetrepresent^d the effect of modules in 
groups 3 and 4-. . 

4 

The results of the tre.atment group/control grdup comparisons are 
<shoWp in Table V. Fpr module group 1, the 1 comparison of multiple-choice 
item module group scores was significant at better than the .05 level. All 
ottyer comparisons were nonsignificant. „ 

Statistical Reliability and Generali2ability of Results 
■Jable VI presents information on treatment group students who. provided 1 
data for the pretest/posttest comparisons upon which the ^alaims of effec- 
tiveness of the VECS modules are based. These individuals represent the 
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TABLE V 






; 


Results of Concrol Grouo Posctest vs. Treacmenc Group Posccesc 


Comoarisons 


i 


















Number 
of Subjects 


Hignesc 
Possible 
Score 


Mean 


Standard 
Deviacion 


— = 

-7, -*esult3 or 


'•hilzix: le-Cko<ee Izsns 










Module Group \ 
z - -2;3H* 


Concrol 
Treatment 


64 
109' 


T 

17 
17 


7.55 
8.41 


2,40 
2,37 


Module Group 2 
c » -0.83 


Control 
Treacmenc 


87 
94 


20 
20 


8.72 
9,04 


2,56 
2,57 


r, "«? ta I "Z on 


Zel--?atirz$ of Ccnvez£r.ce 










Module Grou0 l . Control 
c ■ 0.42' . Treatmenc 


- 62 
108 


72 


/at i c 

47,50 


a Art 
7 , UU 

9,94 


Module GrOup. ST* Concrol 
c - 0.30 Treatment 


j 84 
* 93 


72 
72 


"48.39 
47.99 


^8,97 
8,58 • 


5, *&$ult3 On 


Self-Patinas of attitude 










Module Group I 
t - 1.07 


* •*> 
* % Control \y% 
Treatment 


62 
102 


48 
48 


37.27 

36,45 
c 


4,78 
4180 


* 

Module Group 2 
t - 


Control 
Treatment 


82 
87 


48 
48 

. ^\ 


37.00 
35.68 


5,17 
7.23 


2. Pevulta on 


Vunber of £g&?nz Pr-ofeasioial Develcvrenz 








'Module GTOup I 
t - -0.56 

• 


Control 
Treatment 


62 
98 


6 
6 


2-87 
3.01 


1.38 
U60 


Module Group 2 
t • -0.18 


Octroi 
> Treatment 


83 

84 • 


6 . 
6 " 


2.71 
2-75 


1.41 


• 


- 






f 




Significant at 


the .05 level , 












* % ' 


V 


* 







broad range of educ^ltiptial attainment and experience in Vocational educa- 
tion that would.be possessed' by the population of vocational educators and 
potential vocational educators fqr whom the module^ are Intended t& provide 
training. Because the modules were rested on such a heterogeneous group of 
individuals, the results of the field test shouljli be generalizafile to the 
entire target population. The assertion of the statistical reliability of 
the field te&t results is based on the fact that the field test was con- 
ducted under natural conditions representing the wide variety of conditions 
for which the modules were designed. Each of the classes in which the 
modules were used can be considered a replication of the field test. Data 
from each of uhe replications were combined rather than analyzed separately 
due to the small number of subjects with both pretest and posttest d^fca at 
each site. Nevertheless, because the modules were tested at different, 
times and different places it i*s likely that the results reported are not 
limited to the field test. 



Evidence that Effects 'are Attributable to the intervention 

- 1 ■■ r i —i ■ p — 1 , 

Table VII presents information on the treatment group and control group 
students who provided -data for the comparisons that reinforced the claims 
of effectiveness for module groups 1 and 2. In general, Table VII shows 
that the two groups were quite similar. Where differences in educational 
attainment and experience exist, they usually favor the control group, thus 

* 

reducing the likelihood that differences in the composition of the two 
groups could have biased the field test results in favor of module effec- ^ 
tiveness. The previously reported, nonsignificant comparison of the aver'- " 
age number of professional development experiences recently experienced by 
the two groups supports this conclusion. 
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. , TAHI.E VI 

Characteristics of Students in the £rje^H^ja Jirj>uj> 







Sex 


Degree 


Credential 


"\ Previous Experience 
\ In Vocational Ed 




Current Poslrt 


on 






No. 


X 
M 


X 

V 


% 

Assoc. 


X 

Bach. 


Must . 


• 

X 

Dr. 


X 

Teach- 
ing* 


X 

Adnlnls- 
t rat Ive 


X 

Teacher 


X 

Adminis- 
trator 


X 

Curric- 
ulum 

Devel- 
op* r 


X 

'IV. m hi* 


X 

Aduiuls- 

l 1 ,lt ill' 


X 

Curric- 
ulum 

Devel- 
oper 


X 

Undo r- 
grud. 
Stnd.nl 


X 

Grad, 

st tni«*iir 


Croup I 


96 


44 


56 


5 


53 


27 


1 


'67 


7 


59 


9 


9 


57 


14 


9 


21 


46 


Module 
Croup 2 


78 


47 


53 


23 


'^47 


26 


1 


74 


5 


53 


5 


13 


55 


9 


15 


26 




Modu le 
Croup 3 


69 


46 


54 


13 


45 


33 


1 


78 


13 


67 


14 


23 


62 


14 


19 


6 


41 


Module 
Croup 4 


68 


41 


59 


6 


44 


44 


4 


75 


9 


46 


w 

12 


18 


51 


15 


15 


1 

-> 


54 



I 

N> 
U> 
I 
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. v TABLE VII *• 

Characteristics of Students Providing Data to the Treatment Group/Contro l Croup Comparisons 



No. 


Sex 


Degree 


Credent ial 


Previous Experience 
in Vocational Ed 


Current Position 


X X 
M F 


X X X % X 
Assoc. Bach* Mast* Dr. 


X X 
Teach- Adminis- 
tng trative 


X^ 
Curried 
X ulum 
^ Admlnis- Devel- 
Teacher trator oper 


* X 

Curric- X 
X ulum Under- X 
X • Adialuls- Devel- grad. Crad. 
Teacher trator oper Student Student 


a.' Modul *t Group I 


Tre.it- 
client 

Students 109 

Control** 
Students 64 


44 56 
38 63 


» 

6 51 26 1 

8 44 42 2 
i 


62 6 
94 23 


57 8 8 
67 11 2 


56 13 9 24 44 
61 20 6. 14 53 


*>. "child oYr 


Up ?. f 




TriML- 
in cut 

Student 8 94 

Control 
Students 87 


47 53 
40^ 60 


24 48 22 2 
6 45 43 3 


70 4 
93 25 


52 4 10 
70* 14 2 


53 - 9 13 30 39 

63 20 * 8 . 10 5S . 
— — — « ■ ■ * . I 



(RUE FOft TABLES Vl AND VII: Percentages do not always add to 100% due to rounding. mi sal ng data, or marking more than one category for 
gree. Credential, Previous Experience, or Current Position. 
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The strongest evidence that the field test results were due to module 
study comes from factors in the design of the field test, however. Since 
the field test was conducted at different times at a number of- sites, the 
effects of events other than module study would tend to average out. Prac- 
tice effects on multiple-choice items are unlikely since different items 
comprised the pretest and posttest instruments. Practice Effects on short- 
answer items and maturation effects are unlikely to have caused field test 
results because the tests are so closely tied to the content of the module 
that gains resulting frpm other sources are improbable. 

Educational Significance of Results 

" * The results of the national fi^Ld test demonstrated that the modules 

» 

increase knowledge of topics necessary to the successful performance of 
skills central to the VECS role. They also tended to increase peoples 1 
^confidence in their ability to perform these skills. As vocational educa- 
tion expands and changes, more VECS will be needed to create and maintain 

f « 
high quality curricula. VECS will receive training under many different 

circumstances. The VECS modules provide a low cost, soundly developed 
' resource that is adaptable to a variety of needs. The modules summarize 4 
and organize a great body of material permitting VECS training to be con- 
ducted Systematically *at many different types of locations** 
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Chapter IV 
" DISSEMINAT ION OF THE MATERIALS 

In addition to attending numerous meetings and conventions of profes- 
sional associations in order to inStoduce vocational educators* to the VECS 
materials, AIR staff cpnducted a series of fechnical assistance forums and 
dissemination workshops at locations across the country • At the six one-day 
forums, field test instructors and site coordinators gave detailed" accounts 
of how to use the VECS modules in various settings ♦ Approximately 250 
vocational educator^ from 53 states and territories attended the workshops, 
which vtare held in- the following locations': Los Alamitos, California; 
^Oklahoma City, Oklahoma; Atlanta, Georgia; Newton, Massachusetts; Columbus, 
Ohio; and Salt Lake City, Utah. The attendees included state-level voca- 
tional education administrators,, state liaison representatives of the 
National Network for Curriculum Coordination in Vocational Technical Educa- 
tion (NNCCVT?), college and ^university, faculty, and administrators from 
large school districts. The particpants left the workshops with plans for 
using and disseminating the materials throughout their states. 

9 / * 

In September, the vocational educators who attended the dissemination 
.workshops were sent a^number of summaries and reports that came out of the 

Workshops for t£eir use in following through on their staterlevel dissemi- 

/ 

nation plans. These materials included: ^, 

• a roster of workshop participants, 

• a summary of participants 1 reactions to the workshops, 

a listing of considerations in disseminating the modules, 

I 



• a journal article- that discusses field test findings, and 

• A copy of the final issue of the VECS newsletter. 

A number of journal articles- were written to inform vocational educa- 
tors of the VECS materials including: / 

• Claudy, C. B. , & Hamilton, & A. Results of the national field test 

of the vocational education curriculum specialist materials. 
Journal of Vocational Education Research, Summer 1981. 



• Claudy, C. B. , Blank, W . E. , & Hamilton, J. A. Field testing the 
VECS modules in Florida. Florida Vocational Journal, in press. 



• Claudy, C. B. , & Hamilton, J. A. Modules designed for developing VE 
curriculum experts. American Technical Education Association 
- Journal, 1981, 9(2), 8. 



• Claudy, C. B.', Hamilton, J. A., Kelly, J. F., Cummings, J. M., & 
Capello, F. L. Field testing the VECS modules in New Jersey. 
VEANJ Journal, September 1981. 



• Claudy,^ C. B. , Hamilton, J. A., & McDonald, B. SIU's off campus 
field test of the VECS modules. Illinois Vocational Education 
Journal, in press. 



• Kaplan,. C. B., Hamilton, J. A., & tyheeler, J 4 D. The role of the 

vocational education curriculum specialist ''in fostering industry- 
education cooperation. Journal of Industry-Education Cooperation , 
1980, 1(2), 19-25. 

• Training vocational educators as curriculum developers. Voc Ed 

Insider , 1979, 54(5), 32d.. 

An agreement with the East Central Network Curriculum Center , a member 
of the NNCCVTE, to print and distribute the VECS materials on a cost recov- 
ery basis was approved by the Copyright Authorization Office, U.S. Depart- 
ment of Education. It was agreed that a complete set" of tpe materials can 

/ 
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be purchased for $35.00. Included in the complete set are the 16 modules, 
an instructor's guide, an aydio tape cassette, and a report of the field 
test. A complete set, or portions of it, can be ordered directlyj^fr^m: 



East Central Network Curriculum Center 
Sangamon State University, E-22 
Springfield, IL § % 2708 





APPENDIX A 

List of' Materials Produced Under Contract 300-78M)562 



Modules 



Title 



Vocational Educators and Curriculum Management 
The Scope of Vocational Education 
Organization of Vocational Education 
Legislative Mandates for Vocational Education 
Priorities in Vocational Education 

Vocational Education jfor Students with Special NeedS 

/ 

Vocational Needs Assessment and Curriculum Development 

~-gqf iduct : l ng -^ -sk--Ara — 

Instructional Objectives 

Selecting Instructional Strategies and Assessing 

Student Achievement / 

Relating Leara|p£ Differences and Instructional 
Methods 

Selecting and Preparing Instructional Materials 
Evaluating Vocational Education Curricula 

Conducting Follow-up Studies and Communicating 
( Evaluation Results 



Managing Vocational Education Programs 
Preparing for Curriculum Change 
Staff Development 



Guide 



Title 



Using the VECS Modules: A Guide for Instructors 
and Administrators 



Author (s) 

Jeanette D. Wheeler 

Jeknette D. Wheeler 

Jeanette D. Wheeler 

Jeanette D. Wheeler 

Jeanette D. Wheeler 

Judith A- Appleby 
Jeanette D. Wheeler / 

Judith A, Appleby 

Judith A, Appleby 

Judith A, Appleby 

Carol B. Kaplan 

Judith A. Appleby 

Jean Wolman 
Carolyn B. Claudy 

Carolyn McFarlane 
Carolyn B. Claudy 

Judith A. Appleby 

/ Judith A. Appleby 

Barbara Pletcher 



Author (s) 



Carolyn B. Claudy 
Jeanette D. Wheeler j 
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Audio Casette Tape 
Title 4 ' 

Using the VECS Modules: Comments of Vocational 
Educators 
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Author (s) 



Jurfeen M. Wolff 



Convention Papers 

————————— ^ 

Title 

The National Field Test of the 
Vocational Education Curriculum, 
Specialist Materials 



Convention/ Date 

American Vocational 
Association, 12/ 79 



Author (s) 

Jack A, Hamilton 
Carolyn B. Claudy 



Results of the National Field 
Test of; the Vocational Education 
Curriculum Specialist Materials 



American Vocational 
Assocation. 12/80 



Carolyn B. Claudy 
Jack A, Hamilton 



Making Vocational Curriculum 
Responsive to Student Needs 



American Personnel & 
Guidance Associa- 
tion. 4/80 < ' 



Jack A, Hamilton 
Carolyn B. Claudy 
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Journal Articles 



Title 



Training Vocational Educators as 
Curriculum Developers 

The Role of the Vocational 
Education Curriculum Specialist 
in fostering industry-education 
cooperation 

$IU's Off-Campus Field Test of the 
VECS Modules 



festal 



Field Testing the VECS Modules 
in New Jersey 

Modules Designed for Developing 
VE Curriculum Experts 

Field Testing the VECS Mobiles 
in Florida _ __: 



Results of the National Field 
Test of the Vocational Education 
Curriculum Specialist Materials 



Journal /Date 
VocEd Insider 



Journal of 
Industry-Educa- 
tion Cooperation 



Illinois Vocational 
Education Journal* 



J Journal 



ATEA Journal 



Florida Vocational 
Journal 



Journal <$f Vocation- 
al Education Research 



Jbint Dissemination Review Panel Submission 

Date 



Title 

Submission to the JDRP 



11/80 



Author (s) 

Jack A, Hamilton 
Carol B. Kaplan 

Carol B. Kaplan 
Jack A, Hamilton 
Jeanette D, Wheeler 



Carolyn B. Claudy 
Jack A, Hamilton 
"Bruce McDonald 

Carolyn B. Claudy 
Jack A, Hamilton 

Carolyn B. Claudy 
Jack A, Hamilton 

Carolyn B. Claudy 
William t. Blank 
Jack A, Hamilton 

Carolyn B. Claudy 
Jack A, Hamilton 



Author (?) 

Carolyn B. Claudy 
Jack A. Hamilton 
Steven M. Jung 
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1* 


VECS Newsletters 


t 




* 


* 


Title 


Date 




Editor (s) 




VECS newsletters, three issues 

* 


1978-1979 ' 




Carol B. Kaplan ; H 




VECS newsletters, three issues 


1979-1980 




Barbara Pletcher ' 
Emily Caippbell 




VECS newsletters, three issues 


1980-1981 




Emily Campbell 
Carolyn B. Claudy 


• 


Technical Reports 


- 








L Title 


Date 




Author (s) 




Plan for Familiarization ^ 
Program (1) 


11/78 


i 


Carol B. Kaplan 
Jack A. Hamilton 

1 


# 


Guideline? for Revision: 
Recommendations' for Inte- 
grating and Updating WSU and 
AIR VECS Materials (2) 


3/79 


_ 1 


Judith "A, Appleby 
Jack A. Hamilton 
Carol B. Kaplan 
Jeanette D. Wheeler - 




Study Design and Analysis 
Plan (3) 


4/79 




^ Steven M. Jung 3 
Carolyn B. Helliwell 
Jack A. Hamilton 


f 


Plan for Familiarization 
Program-Phase II (4) 


11/79 




Car<j>l B. Kaplan 
Jack A. Hamilton 




Plan for Familiarization 
Program-Phase III (5) 


2/81 




Jack *A. Hamilton 




Written Summary of -Consul- 
tations with Staff at 
Module-Using Sites (6) . 


5/81 




Jack A. Hamilton 

i 




Written Summary of Workshop 
Accompli. shmerits XT) 
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